<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]-->
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 1.5.8">
<title>Concourse Pipeline (Kubernetes)</title>
<link rel="stylesheet" href="css/spring.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
</head>
<body class="article toc2 toc-left">
<div id="header">
<div id="toc" class="toc2">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel1">
<li><a href="#concourse-pipeline-k8s">Concourse Pipeline (Kubernetes)</a>
<ul class="sectlevel2">
<li><a href="#step-by-step-k8s">Step-by-step</a></li>
<li><a href="#concourse-start-k8s">Concourse in K8S (Kubernetes)</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="concourse-pipeline-k8s"><a class="link" href="#concourse-pipeline-k8s">Concourse Pipeline (Kubernetes)</a></h2>
<div class="sectionbody">
<div class="admonitionblock important">
<table>
<tr>
<td class="icon">
<i class="fa icon-important" title="Important"></i>
</td>
<td class="content">
In this chapter, we assume that you deploy your application
to Kubernetes PaaS
</td>
</tr>
</table>
</div>
<div id="concourse-k8s" class="paragraph">
<p>The Cloud Pipelines repository contains opinionated
Concourse pipeline definitions. Those jobs form an empty pipeline and an
opinionated sample pipeline that you can use in your company.</p>
</div>
<div class="paragraph">
<p>The following projects take part in the <code>microservice setup</code> for this demo:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics-kubernetes">Github Analytics</a>: The application that has a REST endpoint and uses messaging&#8201;&#8212;&#8201;part of our business application.</p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-webhook-kubernetes">Github Webhook</a>: Project that emits messages that are used by Github Analytics&#8201;&#8212;&#8201;part of our business application.</p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-eureka">Eureka</a>: Simple Eureka Server. This is an infrastructure application.</p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics-stub-runner-boot">Github Analytics Stub Runner Boot</a>: Stub Runner Boot server to be used for tests with Github Analytics and uses Eureka and Messaging. This is an infrastructure application.</p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="step-by-step-k8s"><a class="link" href="#step-by-step-k8s">Step-by-step</a></h3>
<div class="paragraph">
<p>If you want only to run the demo as far as possible by using PCF Dev and Docker Compose, do the following:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><a href="#concourse-fork-k8s">Fork repos</a></p>
</li>
<li>
<p><a href="#concourse-start-k8s">Start Concourse and Artifactory</a></p>
</li>
<li>
<p><a href="#concourse-pipeline-fly-k8s">Setup the <code>fly</code> CLI </a></p>
</li>
<li>
<p><a href="#concourse-pipeline-credentials-k8s">Setup your <code>credentials.yml</code> </a></p>
</li>
<li>
<p><a href="#concourse-pipeline-build-k8s">Setup the pipeline </a></p>
</li>
<li>
<p><a href="#concourse-pipeline-run-k8s">Run the <code>github-webhook</code> pipeline</a></p>
</li>
</ol>
</div>
<div class="sect3">
<h4 id="fork-repos-k8s"><a class="link" href="#fork-repos-k8s">Fork Repositories</a></h4>
<div id="concourse-fork-k8s" class="paragraph">
<p>Four applications compose the pipeline:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-webhook-kubernetes">Github Webhook</a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics-kubernetes/">Github Analytics</a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-eureka">Github Eureka</a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics-stub-runner-boot-classpath-stubs">Github Stub Runner Boot</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>You need to fork only the following repositories, because only then can you tag and push the tag to the repository:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-webhook-kubernetes">Github Webhook</a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/github-analytics-kubernetes/">Github Analytics</a></p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect2">
<h3 id="concourse-start-k8s"><a class="link" href="#concourse-start-k8s">Concourse in K8S (Kubernetes)</a></h3>
<div class="paragraph">
<p>The simplest way to deploy Concourse to K8S is to use <a href="https://github.com/kubernetes/helm">Helm</a>.
Once you have Helm installed and your <code>kubectl</code> is pointing to the
cluster, run the following command to install the Concourse cluster in your K8S cluster:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ helm install stable/concourse --name concourse</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Once the script is done, you should see the following output</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">1. Concourse can be accessed:

  * Within your cluster, at the following DNS name at port 8080:

    concourse-web.default.svc.cluster.local

  * From outside the cluster, run these commands in the same shell:

    export POD_NAME=$(kubectl get pods --namespace default -l "app=concourse-web" -o jsonpath="{.items[0].metadata.name}")
    echo "Visit http://127.0.0.1:8080 to use Concourse"
    kubectl port-forward --namespace default $POD_NAME 8080:8080

2. Login with the following credentials

  Username: concourse
  Password: concourse</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Follow the steps and log in to Concourse under <a href="http://127.0.0.1:8080" class="bare">http://127.0.0.1:8080</a>.</p>
</div>
<div class="sect3">
<h4 id="deploying-artifactory-to-k8s"><a class="link" href="#deploying-artifactory-to-k8s">Deploying Artifactory to K8S</a></h4>
<div class="paragraph">
<p>You can use Helm also to deploy Artifactory to K8S, as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ helm install --name artifactory --set artifactory.image.repository=docker.bintray.io/jfrog/artifactory-oss stable/artifactory</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>After you run this command, you should see the following output:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">NOTES:
Congratulations. You have just deployed JFrog Artifactory Pro!

1. Get the Artifactory URL by running these commands:

   NOTE: It may take a few minutes for the LoadBalancer IP to be available.
         You can watch the status of the service by running 'kubectl get svc -w nginx'
   export SERVICE_IP=$(kubectl get svc --namespace default nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
   echo http://$SERVICE_IP/

2. Open Artifactory in your browser
   Default credential for Artifactory:
   user: admin
   password: password</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Next, you need to set up the repositories.</p>
</div>
<div class="paragraph">
<p>First, access the Artifactory URL and log in with
a user name of <code>admin</code> and a password of <code>password</code>.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/artifactory_quick_setup.png" alt="artifactory quick setup">
</div>
<div class="title">Figure 1. Click on Quick Setup</div>
</div>
<div class="paragraph">
<p>Then, click on Maven setup and click <code>Create</code>.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/artifactory_maven_repo.png" alt="artifactory maven repo">
</div>
<div class="title">Figure 2. Create the <code>Maven</code> Repository</div>
</div>
</div>
<div class="sect3">
<h4 id="concourse-pipeline-fly-k8s"><a class="link" href="#concourse-pipeline-fly-k8s">Setup the <code>fly</code> CLI</a></h4>
<div class="paragraph">
<p><a id="fly"></a> If you go to the Concourse website you should see something resembling the following:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/running_concourse.png" alt="running concourse">
</div>
</div>
<div class="paragraph">
<p>You can click one of the icons (depending on your OS) to download <code>fly</code>, which is the Concourse CLI. Once you download that (and maybe added it to your PATH, depending on your OS) you can run the following command:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">fly --version</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>If <code>fly</code> is properly installed, it should print out the version.</p>
</div>
</div>
<div class="sect3">
<h4 id="concourse-pipeline-credentials-k8s"><a class="link" href="#concourse-pipeline-credentials-k8s">Setup your <code>credentials.yml</code></a></h4>
<div class="paragraph">
<p>We made a sample credentials file called <code>credentials-sample-k8s.yml</code>
prepared for <code>k8s</code>. You can use it as a base for your <code>credentials.yml</code>.</p>
</div>
<div class="paragraph">
<p>To allow the Concourse worker&#8217;s spawned container to connect to the
Kubernetes cluster, you must pass the CA contents and the
auth token.</p>
</div>
<div class="paragraph">
<p>To get the contents of CA for GCE, run the following command:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ kubectl get secret $(kubectl get secret | grep default-token | awk '{print $1}') -o jsonpath='{.data.ca\.crt}' | base64 --decode</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>To get the auth token, run the following command:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ kubectl get secret $(kubectl get secret | grep default-token | awk '{print $1}') -o jsonpath='{.data.token}' | base64 --decode</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Set that value under <code>paas-test-client-token</code>, <code>paas-stage-client-token</code>, and <code>paas-prod-client-token</code></p>
</div>
</div>
<div class="sect3">
<h4 id="concourse-pipeline-build-k8s"><a class="link" href="#concourse-pipeline-build-k8s">Build the pipeline</a></h4>
<div class="paragraph">
<p>After running Concourse, you should get the following output in your terminal:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ export POD_NAME=$(kubectl get pods --namespace default -l "app=concourse-web" -o jsonpath="{.items[0].metadata.name}")
$ echo "Visit http://127.0.0.1:8080 to use Concourse"
$ kubectl port-forward --namespace default $POD_NAME 8080:8080
Visit http://127.0.0.1:8080 to use Concourse</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Log in (for example, for Concourse running at <code>127.0.0.1</code>&#8201;&#8212;&#8201;if you do not provide any value, <code>localhost</code> is assumed). If you run this script, it assumes that either <code>fly</code> is on your <code>PATH</code> or that it is in the same folder as the script:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ fly -t k8s login -c http://localhost:8080 -u concourse -p concourse</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Next, run the following command to create the pipeline:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ ./set_pipeline.sh github-webhook k8s credentials-k8s.yml</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect3">
<h4 id="concourse-pipeline-run-k8s"><a class="link" href="#concourse-pipeline-run-k8s">Run the <code>github-webhook</code> Pipeline</a></h4>
<div class="paragraph">
<p>The following images show the various steps involved in runnig the <code>github-webhook</code> pipeline:</p>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_login.png" alt="concourse login">
</div>
<div class="title">Step 1: Click <code>Login</code></div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_team_main.png" alt="concourse team main">
</div>
<div class="title">Step 2: Pick <code>main</code> team</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_user_pass.png" alt="concourse user pass">
</div>
<div class="title">Step 3: Log in with <code>concourse</code> user and <code>concourse</code> password</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_pipeline.png" alt="concourse pipeline">
</div>
<div class="title">Step 4: Your screen should look more or less like this</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/start_pipeline.png" alt="start pipeline">
</div>
<div class="title">Step 5: Unpause the pipeline by clicking in the top lefr corner and then clicking the <code>play</code> button</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/generate_version.png" alt="generate version">
</div>
<div class="title">Step 6: Click 'generate-version'</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/run_pipeline.png" alt="run pipeline">
</div>
<div class="title">Step 7: Click <code>+</code> sign to start a new build</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/concourse_pending.png" alt="concourse pending">
</div>
<div class="title">Step 8: The job is pending</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/job_running.png" alt="job running">
</div>
<div class="title">Step 9: Job is pending in the main screen</div>
</div>
<div class="paragraph">
<p>&#160;
&#160;</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/concourse/running_pipeline.png" alt="running pipeline">
</div>
<div class="title">Step 10: Job is running in the main screen</div>
</div>
</div>
</div>
</div>
</div>
</div>
<link rel="stylesheet" href="js/highlight/styles/atom-one-dark-reasonable.min.css">
<script src="js/highlight/highlight.min.js"></script>
<script>hljs.initHighlighting()</script>
</body>
</html>